Papers with Levenshtein Transformer
EDITOR: An Edit-Based Transformer with Repositioning for Neural Machine Translation with Soft Lexical Constraints (2021.tacl-1)
Copied to clipboard
| Challenge: | Empirically, EDITOR uses soft lexical constraints more effectively than the Levenshtein Transformer while speeding up decoding dramatically compared to constrained beam search. |
| Approach: | They propose an Edit-Based TransfOrmer with Repositioning that integrates lexical preferences into output sequences by iterative editing hypotheses. |
| Outcome: | The proposed model uses soft lexical constraints more effectively than the Levenshtein Transformer while speeding up decoding dramatically compared to constrained beam search. |
Reinforcement Learning for Edit-Based Non-Autoregressive Neural Machine Translation (2024.naacl-srw)
Copied to clipboard
| Challenge: | Non-autoregressive (NAR) language models have a performance gap due to the large decoding space and difficulty in capturing dependency between target words accurately. |
| Approach: | They propose to use reinforcement learning to enhance the performance of edit-based NAR models by using stepwise reward maximization and episodic reward maximisation. |
| Outcome: | The proposed model outperforms autoregressive models in the evaluation of an edit-based model. |
Benchmarking Diffusion Models for Machine Translation (2024.eacl-srw)
Copied to clipboard
| Challenge: | Diffusion models have shown great potential on many generative tasks, but their application to natural language processing (NLP) is still a less explored direction. |
| Approach: | They adapt two diffusion-based text generation models, Diffusion-LM and DiffuSeq, to perform machine translation. |
| Outcome: | The proposed models struggle more on long-range dependencies than other models. |
Integrating Translation Memories into Non-Autoregressive Machine Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | Non-autoregressive machine translation (NAT) has made great progress, but most studies focus on standard translation tasks. |
| Approach: | They propose to train an edit-based NAT model with a Translation Memory (TM) they propose to modify the data presentation and introduce an extra deletion operation to reduce decoding load. |
| Outcome: | The proposed model performs on par with an autoregressive approach while reducing the decoding load. |
Towards Example-Based NMT with Multi-Levenshtein Transformers (2023.emnlp-main)
Copied to clipboard
| Challenge: | Retrieval-augmented machine translation (RAMT) is attracting growing attention . it is assumed to implement some form of domain adaptation . |
| Approach: | They propose a retrieval-augmented version of the Levenshtein Transformer to make it more transparent . they propose to perform training and inference in this model, based on multi-way alignment algorithms and imitation learning. |
| Outcome: | The proposed architecture improves translation performance and improves consistency of translations compared to previous models. |
How Does Distilled Data Complexity Impact the Quality and Confidence of Non-Autoregressive Machine Translation? (2021.findings-acl)
Copied to clipboard
| Challenge: | Prior work suggests that distilled training data is less complex than manual translations. |
| Approach: | They propose to use sequence-level knowledge distillation to match autoregressive models' translation quality. |
| Outcome: | The proposed model can match translation quality of autoregressive models with distilled training data. |
Levenshtein Training for Word-level Quality Estimation (2021.emnlp-main)
Copied to clipboard
| Challenge: | a novel scheme to perform word-level quality estimation is proposed for word-based quality estimation . authors propose a two-stage transfer learning procedure on augmented and human data . a Levenshtein Transformer can learn to post-edit without explicit supervision. |
| Approach: | They propose a novel scheme to use a Levenshtein Transformer to perform word-level quality estimation. |
| Outcome: | The proposed method performs better under data-constrained and unconstrained conditions than existing methods. |